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^ Abstract 

kjr) GALPROP is a numerical code for calculating the galactic propagation of relativistic charged particles and the diffuse emissions 
^5 produced during their propagation. The code incorporates as much realistic astrophysical input as possible together with latest 
theoretical developments and has become a de facto standard in astrophysics of cosmic rays. We present GALPROP WebRun, a 
service to the scientific community enabling easy use of the freely available GALPROP code via web browsers. In addition, we 
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introduce the latest GALPROP version 54, available through this service. 
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1. Introduction 

A large number of outstanding problems in physics and astro- 
physics are connected with studies of cosmic rays (CRs) and the 
associated diffuse emissions (radio, microwave, X-rays, y-rays) 
produced during their propagation. Among these problems are 
indirect searches for dark matter (DM), the origin and propaga- 
tion of CRs, particle acceleration in putative CR sources - such 
as supernova remnants - and the interstellar medium (ISM), 
Cosmic rays in other galaxies and the role they play in galactic 
evolution, studies of our local Galactic environment, CR propa- 
gation in the heliosphere, the origin of the 511 keV line from the 
Galactic Center, the origin of the extragalactic diffuse emission, 
as well as many others. New or improved instrumentation to 
explore these open issues are operating or under development. 
Low-energy CR detectors on spacecraft and balloons, such as 
ACE, TIGER, the Voyagers 1 and 2, provide data on isotopic 
composition, and are complemented by the currently on-orbit 
PAMELA experiment, which is designed to measure antipro- 
tons as well as light nuclei, electrons, and positrons above 100 
GeV Elemental spectra are provided up to the TeV range by the 
experiments CREAM, ATIC, and TRACER, while PPB-BETS, 
CALET, and the Fer/ra'-LAT provide measurements of elec- 
trons. Instruments such as INTEGRAL, Fermi— LAlT, HESS, 
MILAGRO, MAGIC, and VERITAS cover from hard X-rays 
to y-rays up to TeV energies. A number of other experiments 
are in the research phase, under construction, or about to be 
launched, for example, AMS, OASIS, HAWK, and CREST. For 
a recent review see [35 1. 



The complex nature of these scientific goals, such as, e.g., 
the detection of a weak DM signal on top of the intense diffuse 
y-ray emission produced by CRs interacting with the ISM, or 
the study of K-capture isotopes and electrons in CRs, requires 
reliable and detailed calculations using a numerical model. All 
of the latest developments of astrophysics, and particle and nu- 
clear physics, play a role in addressing these questions: the lat- 
est developments in CR acceleration and transport mechanisms, 
detailed maps of the three-dimensional Galactic gas distribu- 
tion, detailed studies of the interstellar dust, radiation field, and 
magnetic field, as well as the Local Bubble, and new particle 
and nuclear cross section data and codes. Achieving these sci- 
entific goals requires a realistic, yet simple to access and use, 
model with most of the technical details left to experts. Alter- 
natively, sophisticated users must have access to the full source 
code and documentation to easily allow them to modify the 
code as suits their purpose. 

GALPROP is a numerical code for calculating the propaga- 
tion of relativistic charged particles and the diffuse emissions 
produced during their propagation. The GALPROP project has 
been running since the mid-1990s, and since then it has evolved 
into a sophisticated, efficient, configurable tool for high energy 
astrophysics 156*1 . The code incorporates as much realistic as- 
trophysical input as possible together with latest theoretical de- 
velopments. GALPROP calculates the propagation of CR nu- 
clei, antiprotons, electrons and positrons, and computes diffuse 
y-rays and synchrotron emission in the same framework. Each 
run of the code is governed by a configuration file allowing 
the user to specify and control many details of the calculation. 
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Figure 1 : The GALPROP WebRun service allows users to create a GALPROP model using a web browser, validate the model parameters to reduce the likelihood 
of misconfigured runs, and execute the GALPROP code on a dedicated computing cluster. The service is accessible via http://galprop.stanford.edu/webrun/ 



Thus, each run of the code corresponds to a potentially different 
"model". The code is written mainly in C++ along with some 
well-tested routines in Fortran 77. 

Formalism and results of the GALPROP code can be found 

in papers (DI2l3ffll30l21IHll3ISaiIIllI21II3IIllI3IISllI2l 

DEED, conference proceedings Il20l |2T1 [22l l23l [24l l25l [26ll27l 
|28l|29][30j[3Tl[32l, and reviews lT33ll34ll35l. 

Until now, the only way to use the GALPROP code has been 
downloading the source code from the project web site, in- 
stalling and running it on a user's machine. Besides occasional 
problems with the installation process, users often experienced 
difficulties configuring the GALPROP runs. 

2. The GALPROP WebRun service 

The GALPROP WebRun service presented here enables any 
member of the scientific community to use the most recent 
version of GALPROP via a web browser. Installation of the 
code locally is not required with the calculations performed on 
a dedicated computing cluster employing AMD Opteron 6174 
CPUs managed by the GALPROP team (cluster specifications 
are given in Section [4]). In addition, using the input web forms 
of WebRun allows the input parameters to have sanity checks 
applied via rules written by the GALPROP team, thus minimiz- 
ing the risk of misconfigured GALPROP runs (see Figure[T]for 
an example). Each user has a queue that can be used for batch 
jobs of multiple GALPROP runs, where the user load is dy- 
namically distributed across the available compute nodes. We 



also provide the latest stable release of GALPROP along with 
earlier tagged releases to allow cross-checking of results across 
different versions. 

The results of a GALPROP run (CR distributions and diffuse 
emission sky maps) are written out in FITS files readable by 
common astronomical software ll36ll . Some of this data (e.g., 
energy spectra of CR isotopes, abundances of CR species, and 
spectra of diffuse emissions) can be plotted using the WebRun 
interface. The output of a GALPROP run generally contains 
much more usable information than given by the built-in plot 
templates in GALPROP WebRun, so users are encouraged to 
build their own plotting routines that use the data files produced 
by GALPROP. 

WebRun stores the results of each run along with the 
GALPROP configuration file on the web server from where the 
author of the results can download them as a compressed 'tar' 
archive. Sharing the results is also made easy by WebRun: the 
author of the run can publish the URL of the tar archive, which 
will make it possible for anyone to download and peruse the 
results. The URLs of user-generated WebRun archives are en- 
coded with a string of 16 random characters. This string acts as 
a key to the run known only to the user that generated it, unless 
he or she decides to share or publish the URL. 

The GALPROP WebRun service can be accessed on the 
World Wide Web Il37l . Registration is required to use WebRun, 
which is done via a form at the WebRun URL. 
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3. New features in the latest GALPROP version 54 

Simultaneously with the release of the WebRun service, we 
present an improved version the GALPROP code (v54). This 
version is available to the scientific community only via the 
GALPROP WebRun site. The new features of the code include: 

• Shared-memory parallel support with OpenMP to take ad- 
vantage of multi-processor machines; 

• Memory usage optimization; 

• Implementation of the HEALPix output of gamma-ray and 
synchrotron skymaps. The HEALPix format [38 1 is a stan- 
dard for radioastronomy applications, as well as for such 
instruments as WMAP, Planck etc.; 

• Implementation of the MapCube output for compatibility 
with Fermi -LAT Science Tools software |39|; 

• Implementation of gamma-ray skymaps output in Galac- 
tocentric rings to facilitate spatial analysis of the Galactic 
diffuse gamma-ray emission; 

• More accurate line-of-sight integration for computing dif- 
fuse emission skymaps; 

• 3D modeling of the Galactic magnetic field, both regular 
and random components, with a range of models from the 
literature, extensible to any new model as required; 

• Calculations of synchrotron skymaps on a frequency grid, 
using both regular and random magnetic fields; 

• Improved gas maps, which are computed using recent H I 
and CO (H2 tracer) surveys l40l |4T1 . with more precise 
assignment to Galactocentric rings; 

• A new calculation of the Galactic interstellar radiation 
field using the FRaNKIE code (Fast Radiation transport 
Numerical Kode for Interstellar Emission, as described in 
|[T7lD and implementation of the corresponding changes in 
GALPROP; 

• Considerably increased efficiency of anisotropic inverse 
Compton scattering calculations; 

• GALPROP code is compiled to a library for easy linking 
with other codes (e.g., DarkSUSY 02, SuperBayeS ll43l ); 

• Numerous bug fixes and code-style improvements; 

• Improved configuration management via the GNU auto- 
tools. Multiple *NIX system and compiler targets (gcc, 
intel, llvm, open64) are supported; 

• Bugzilla available at the GALPROP WebRun URL for 
user-submitted bug tracking. 



4. Computing cluster specifications 

WebRun runs on a cluster comprised of a head node, which 
is used as a web and data server, and four compute nodes that 
are connected with Gb ethernet and Infiniband links. The head 
node is 2-way machine using 12-core AMD Opteron 6174 pro- 
cessors with 64 GB of physical memory and 16 TB of redundant 
data storage. Each compute node is a 4-way system using 12- 
core AMD Opteron 6174 processors with 128 GB of physical 
memory and a modest amount of local storage via redundant 
disks (2 x 300 GB mirrored drives) for the operating system 
and other critical files. Data sets and other common files are 
shared via NFS from the head node, as is the common stor- 
age area that run results from the compute nodes are written 
to. Since the current GALPROP parallelization scheme uses 
a shared memory model (via OpenMP), these AMD-based sys- 
tems have enabled us to consolidate computational and memory 
resources within a single unit to enable even high-resolution 3D 
calculations. In addition, these systems allow us to easily ex- 
tend the parallel calculations to CPUs and add-in GPU cards 
using, e.g., OpenCL, or even a hybrid MPI/OpenMP/OpenCL 
scheme (with some rearchitecting) to use more than one com- 
pute node in the future. Similar capabilities were not possible 
with, e.g., a blade-based system given our power and cost-per- 
unit requirements. At any rate, this relatively modest system is 
easily extended dependent on demand for the WebRun service. 

The GALPROP WebRun cluster uses the CentOS E3 Linux 
distribution along with the OSCAR P31 software suite that is 
used to install, configure, and synchronize, the software across 
the cluster. The run queue and resource allocation for user runs 
of GALPROP via WebRun are managed using TORQUE gS) 
in combination with Maui |4"71 . User registration and authen- 
tication in WebRun is linked to a database on the GALPROP 
forum, which based on the open source bulletin board software 
phpBB 11481 . A Bugzilla [49 1 is provided for bug tracking and 
feature requests. The front end of GALPROP WebRun, i.e., the 
browser-based user interface, as well as the back-end routines 
interacting with the cluster software, are a PHP -based custom 
product developed by the GALPROP team. 

GALPROP WebRun calculations submitted via the web in- 
terface are added to a run queue. If enough cores are avail- 
able for computation, the calculation in the queue starts imme- 
diately. Otherwise, the queue manager schedules runs in such 
a way that each WebRun user gets a fair share of resources, 
and available hardware is used at maximum capacity. Cur- 
rently, all GALPROP v54 runs are assigned 12 cores and up 
to 16 GB of memory per run, resulting in typical run times 
for a GALPROP calculation ranging from a few minutes (if 
only 2D cosmic -ray propagation is involved) to several hours 
(if high resolution 3D cosmic ray/diffuse emission calculations 
are performed). The legacy GALPROP (v50) does not sup- 
port multithreading, and each calculation with this version of 
GALPROP is given 1 processor core. Our policy for alloca- 
tion of computing resources may change in the future in order 
to better serve users interested in computationally intensive or 
batch runs. Users are encouraged to provide feedback on their 
experience with GALPROP WebRun to the development team. 
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5. Using the GALPROP WebRun service 

The interface of WebRun consists of a narrow sidebar menu 
on the left and a wider main panel on the right of the screen (see 
Figure[T|. The links in the sidebar menu open in the main panel 
the interface sections 'WebRun Help', 'Configure & Submit', 
'Monitor Queue', 'Download Results' and 'Exchange Runs', 
and the workflow of running and working with a calculation 
follows the sequence in which these sections are ordered. 

When users first log on and visit the WebRun page, they see 
the 'Configure & Submit' section, in which a default GAL- 
PROP model is loaded. The default model does not corre- 
spond to any published results, but serves as a base with most 
parameters set to reasonable values. More models reproduc- 
ing published results are provided as examples accessible via 
the drop-down menu at the top of the main panel. To assist 
the user, in the 'Configure & Submit' section, GALPROP pa- 
rameters are arranged into groups (the groups are navigable via 
scrolling or a shortcut bar at the top of the main panel), and 
some parameters are hidden to simplify navigation. The sidebar 
menu features options 'First-time User Mode' and 'Advanced 
User Mode', with more parameters visible in the latter. In both 
modes, some parameters are hidden or shown depending on val- 
ues of other parameters. For example, when the user changes 
the GALPROP parameter 'gamma_rays' from its default value 
of '0' to T, additional parameters in the 'Gamma-Ray Emis- 
sion' group appear, such as 'skymap_format', which was irrel- 
evant for 'gamma_rays' equal to 0. To see all parameters, the 
user may choose the Advanced User Mode' and check the box 
'Show Inactive Parameters' in the sidebar menu. Some parame- 
ters, such as names of gas and dust maps, debugging-related and 
obsolete parameters, are fixed and cannot be edited in WebRun, 
but can be perused by checking the box 'Show fixed param- 
eters'. At the bottom of the main panel, the button 'Submit' 
saves the user-specified model parameters and sends them to a 
validation routine, which helps prevent misconfigured runs by 
providing detailed warning messages when parameters are out 
of range or have conflicting values. Upon a successful valida- 
tion (or if the user chooses to ignore the warnings), the user will 
be prompted for a confirmation and given the option to down- 
load the formed GALDEF file (i.e., the GALPROP configura- 
tion file with specified parameter values). Detailed explanation 
of the meaning of all parameters is provided in the web form 
and in the GALPROP Explanatory Supplement ll50l . 

When the user confirms the run, it will be submitted to a run 
queue on the GALPROP computing cluster, and the user will be 
taken to the WebRun section 'Monitor Queue' (see Figure |2j. 
The main panel in this section contains: a list of queued and 
running calculations on the left; a configuration viewer panel 
on the right, which displays the configuration file and control 
buttons ('Stop' and 'Import') for the run selected from the list; 
and a real-time monitor at the bottom, which displays the cho- 
sen calculation's running output or the status of the cluster. Pa- 
rameter 'Title', displayed along with the calculation number in 
the list, is very helpful in organizing and identifying one's runs. 
While the user may close the browser and/or log out from the 
GALPROP web site, it will not stop queued or running calcu- 



lations. The runs will be finished, packaged and moved to the 
list in the 'Download Results' section regardless of the user's 
online status. 

Completed runs may be viewed in section 'Download Re- 
sults', interface layout of which is similar to that of the 'Moni- 
tor Queue' section, but there is no monitor panel (see Figure|3j. 
Clicking a run in the list displays its configuration and controls 
in the viewer panel: 'Download', 'View Log', 'Quick Plots', 
'Import' and 'Delete'. The 'Download' button displays the link 
to the archive file with FITS files produced by GALPROP, and 
this link can be shared with the public, yet it is protected from 
unauthorized access by a random 16-character suffix. The run- 
time log produced by the code, included in the archive, can also 
be viewed online using the 'View Log' button. 'Quick Plots', 
a convenient tool illustrated in Figure [4] allows one to take a 
quick look at the results of the run. The 'Import' button is 
useful for making a number of similar calculations. Clicking 
the button takes the user to the 'Configure & Submit' section, 
where the values of all parameters are identical to those in the 
imported run. Deleting the run using the 'Delete' button ir- 
reversibly removes it from the run list and also erases it from 
the hard drive on the server, but the record of that run (user id, 
run id, timing information and exit status) is retained for usage 
statistics collection. 

There are two more functions available in WebRun: batch 
runs and run exchange. The link to batch runs can be found 
in the sidebar menu in the 'Configure & Submit' submenu. To 
execute a batch of runs, the user can submit an archive in ZIP 
or tar format with GALDEF files using this function. Run ex- 
change is instrumental for collaborations, allowing any regis- 
tered user to gain access to the log, Quick Plots and Import ca- 
pability of another user's run, as long as the link to the archive 
has been shared. Both of these functions are documented in the 
help pages and within the interface. 

6. Getting help and providing feedback 

The GALPROP WebRun service includes help pages and 
a FAQ that explains the features of the web interface to the 
GALPROP code. Information about the GALPROP code is 
available from the GALPROP web site (50). A forum on the 
GALPROP web site ED allows registered GALPROP users to 
discuss the code, ask the developers additional questions, and 
provide feedback. 



7. How to acknowledge the use of GALPROP and WebRun 

If GALPROP is used to obtain results for your publication, 
please cite all GALPROP papers relevant to your results and 
the address of the GALPROP web site [52). In addition, if the 
WebRun service is used, also cite the present paper, Vladimirov 
et al., 2010 in Computer Physics Communication. These terms 
may be updated in the future, with up-to-date instructions in the 
'Terms of Use' section of the GALPROP web site J53). 
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Figure 2: GALPROP calculations in WebRun are executed on a dedicated computing cluster at Stanford University. A job queue with multiple simultaneous runs is 
supported. Each calculation is parallelized across several processor cores, and real time running output can be viewed via the WebRun page. 
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Figure 3: GALPROP WebRun users have access to and control over archives with results of all calculations they have performed. Each archive can be shared by 
publishing its URL. Visualization of results is possible within the WebRun window via the 'Quick Plots' function. 
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Figure 4: Quick Plots is an online plotting tool in GALPROP WebRun, capable of parsing the FITS files produced by GALPROP and creating raw text data tables 
and plots using built-in templates. Users are encouraged to develop their own plotting routines for publication purposes using the data produced by Quick Plots as a 
reference. 
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